Tensor Decomposition Algorithm Research Articles

An Adaptive Sampling Strategy for Real-time Anomaly Detection with Unmanned Sensing Vehicles

Unmanned sensing vehicles (USVs) have been widely used for real-time anomaly detection in various applications, including environmental monitoring, precision agriculture, and military surveillance. The USVs collecting data can only provide partial information about the space being monitored. Thus, it is critical to decide where to deploy the USVs at each point in time to maximize the change detection capability, while minimizing deployment costs. This paper proposes an adaptive sampling strategy for real-time anomaly detection with USVs. First, a novel spatio-temporal sequential tensor decomposition algorithm is developed to decompose the high-dimensional data collected by the USVs into three components, a spatial component, a temporal component, and a sparse component, that captures the locations suspicious of change. The spatial and temporal components are used for one-step prediction to guide the adaptive sampling strategy. The strategy is designed to maximize the detection power and control the deployment costs. The main idea is to balance exploration and exploitation by designing a sampling distribution function to decide where to collect data at each acquisition time. The movement of the USVs is controlled by using Voronoi tessellations on the sampling distribution function. The performance of the proposed framework is demonstrated through simulations and case studies.

Fast autoregressive tensor decomposition for online real-time traffic flow prediction

Online real-time traffic flow prediction typically offers better real-time performance than offline prediction. However, existing studies rarely discussed online real-time traffic flow prediction and the balance between prediction accuracy and computational costs. Training and predicting traffic flow data with complex patterns by artificial intelligence models is usually time-consuming. Some high-accuracy statistical learning methods also reach a performance bottleneck in terms of computational speed. Therefore, a Fast Autoregressive Tensor Decomposition (FATD) algorithm is proposed for online real-time traffic flow prediction. First, the historical tensor data is decomposed by Tucker Decomposition into factor matrices and easy-to-compute core tensors, and these core tensors are modeled by Tensor Seasonal Autoregressive Integrated Moving Average (Tensor SARIMA). Second, a future core tensor is predicted by Tensor SARIMA. By the Inverse Tucker Decomposition, the traffic flow data to be predicted is recovered. The experimental results show that the FATD algorithm can reduce the computational cost while maintaining high accuracy. Compared with baselines, the FATD algorithm reduces MAE, RMSE, and the computational cost by approximately 35.04%, 26.86%, and 99.28% on average, respectively.

Simulation-aided infrared thermography with decomposition-based noise reduction for detecting defects in ancient polyptychs

In recent years, the conservation and protection of ancient cultural heritage have received increasing attention, and non-destructive testing (NDT), which can minimize the damage done to the test subject, plays an integral role therein. For instance, NDT through active infrared thermal imaging can be applied to ancient polyptychs, which can realize accurate detection of damage and defects existing on the surface and interior of the polyptychs. In this study, infrared thermography is used for non-invasive investigation and evaluation of two polyptych samples with different pigments and artificial defects, but both reproduced based on a painting by Pietro Lorenzetti (1280/85–1348) using the typical tempera technique of the century. It is noted that, to avoid as far as possible secondary damages done to the ancient cultural heritages, repeated damage-detection experiments are rarely carried out on the test subjects. To that end, numerical simulation is used to reveal the heat transfer properties and temperature distributions, as to perform procedural verification and reduce the number of experiments that need to be conducted on actual samples. Technique-wise, to improve the observability of the experimental results, a total variation regularized low-rank tensor decomposition algorithm is implemented to reduce the background noise and improve the contrast of the images. Furthermore, the efficacy of image processing is quantified through the structural-similarity evaluation.

Reducing the Control Overhead of Intelligent Reconfigurable Surfaces via a Tensor-Based Low-Rank Factorization Approach

Intelligent reconfigurable surfaces (IRS) are becoming an attractive component of cellular networks due to their ability to shape the propagation environment and thereby improve coverage. While IRS nodes incorporate a great number of phase-shifting elements and a controller entity, the phase shifts are typically determined by the cellular base station (BS) due to its computational capability. Since controlling a large number of phase shifts may become prohibitive in practice, it is important to reduce the control overhead between the BS and the IRS controller. To this end, in this paper, we propose a low-rank modeling approach for the IRS phase shifts. The key idea is to represent the IRS phase shift vector using a low-rank tensor approximation model, where each rank-one component is modeled as the Kronecker product of a predefined number of factors of smaller sizes, obtained via tensor decomposition algorithms. We show that the proposed low-rank models drastically reduce the required feedback requirements associated with the BS-IRS control links. Our simulation results indicate that the proposed method is especially attractive in scenarios with a strong line of sight component, in which case nearly the same spectral efficiency is reached as in the cases with near-optimal phase shifts, but with significantly lower feedback overhead.

Studying differential cross section for elastic proton scattering by a tensor model

Previous studies have shown the reliability of the Bayesian Gaussian CANDECOMP/PARAFAC tensor decomposition (BGCP) algorithm in predicting the independent fission yields and evaluating global uncertainty of the data. This work proposes to apply the BGCP algorithm for the purpose of predicting and evaluating the differential cross section for elastic proton scattering. The tensor model is developed by training with the experimental data of elastic proton scattering from the EXFOR database. The Root Mean Square Error is used to evaluate the experimental data and search for potential errors and omissions in EXFOR, in order to identify outlying data of 98Mo, 100Mo, 144Sm and 209Bi. After the revision for the outlying data in EXFOR, the reproduced values for 144Sm are compared with the remaining controversial entries. Besides, the predicted data for 9C are compared with the published data not available in EXFOR. The results demonstrate that the tensor model can make reasonable predictions and has a role to play in evaluating the quality of elastic proton scattering data. The tensor model may become a powerful tool for nuclear data evaluation in EXFOR.

LINK PREDICTION USING TENSOR DECOMPOSITION

In recent years, tensor decomposition has gained increasing interest in the field of link prediction, which aims to estimate the likelihood of new connections forming between nodes in a network. This study highlights the potential of the Canonical Polyadic tensor decomposition in enhancing link prediction in complex networks. It suggests effective tensor decomposition algorithms that not only take into account the structural characteristics of the network but also its temporal evolution. During the process of tensor decomposition, the initial tensor is decomposed into two-way tensors, also known as factor matrices, representing different modes of the data. These factor matrices capture the underlying patterns or relationships within the network, providing insights into the structure and dynamics of the network. For evaluation, we examine a dataset derived from the WSDM. After preprocessing, the data is represented as a multi-way tensor, with each mode representing different aspects such as users, items, and time. Our primary objective is to make precise predictions about the links between users and items within specific time periods. The experimental results demonstrate that our approach significantly improves prediction accuracy for evolving networks, as measured by the AUC.

WormTensor: a clustering method for time-series whole-brain activity data from C. elegans

BackgroundIn the field of neuroscience, neural modules and circuits that control biological functions have been found throughout entire neural networks. Correlations in neural activity can be used to identify such neural modules. Recent technological advances enable us to measure whole-brain neural activity with single-cell resolution in several species including Caenorhabditis elegans. Because current neural activity data in C. elegans contain many missing data points, it is necessary to merge results from as many animals as possible to obtain more reliable functional modules.ResultsIn this work, we developed a new time-series clustering method, WormTensor, to identify functional modules using whole-brain activity data from C. elegans. WormTensor uses a distance measure, modified shape-based distance to account for the lags and the mutual inhibition of cell–cell interactions and applies the tensor decomposition algorithm multi-view clustering based on matrix integration using the higher orthogonal iteration of tensors (HOOI) algorithm (MC-MI-HOOI), which can estimate both the weight to account for the reliability of data from each animal and the clusters that are common across animals.ConclusionWe applied the method to 24 individual C. elegans and successfully found some known functional modules. Compared with a widely used consensus clustering method to aggregate multiple clustering results, WormTensor showed higher silhouette coefficients. Our simulation also showed that WormTensor is robust to contamination from noisy data. WormTensor is freely available as an R/CRAN package https://cran.r-project.org/web/packages/WormTensor.

High Performance Hierarchical Tucker Tensor Learning Using GPU Tensor Cores

Extracting information from large-scale high-dimensional data is a fundamentally important task in high performance computing, where the hierarchical Tucker (HT) tensor learning approach (learning a tensor-tree structure) has been widely used in many applications. However, HT tensor learning algorithms are compute-intensive due to the “ <i>curse of dimensionality</i> ,” i.e., the time complexity grows exponentially with the order of the data tensor. The computation of HT tensor learning algorithms boils down to tensor primitives, which are amenable to computing on GPU tensor cores. Existing work does not support HT tensor learning using GPU tensor cores. There are three main challenges to address: 1) to accelerate tensor learning primitives using GPU tensor cores; 2) to implement the tensor learning algorithms using GPU tensor cores and multiple GPUs; 3) to support large-scale data tensors exceeding the GPU memory capacity. In this paper, we present efficient HT tensor learning primitives using GPU tensor cores and demonstrate three applications. First, we utilize GPU tensor cores to optimize HT tensor learning primitives, including tensor contractions, tensor matricizations and tensor singular value decomposition (SVD). We employ the optimized primitives to optimize HT tensor decomposition algorithms for Big Data analysis. Second, we propose a novel HT tensor layer for deep neural networks, whose training process only involves a forward pass without back propagation. The forward pass consists of tensor operations, thus further exploiting the computing power of GPU tensor cores. Third, we apply the optimized primitives to develop a tensor-tree structured quantum machine learning algorithm <i>tree-tensor network (TTN)</i> . Compared with TensorLy and TensorNetwork on NVIDIA A100 GPUs, our third-order HT tensor decomposition algorithm achieves up to <inline-formula><tex-math notation="LaTeX">$8.92 \times$</tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX">$6.42 \times$</tex-math></inline-formula> speedups, respectively, and our high-order case achieves up to <inline-formula><tex-math notation="LaTeX">$32.67 \times$</tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX">$23.97 \times$</tex-math></inline-formula> speedups, respectively. Our HT tensor layer for a fully connected neural network achieves <inline-formula><tex-math notation="LaTeX">$49.2 \times$</tex-math></inline-formula> compression at the cost of 0.5% drops in accuracy and <inline-formula><tex-math notation="LaTeX">$1.42 \times$</tex-math></inline-formula> speedup compared with the implementation on CUDA cores; for the AlexNet, our HT tensor layer achieves <inline-formula><tex-math notation="LaTeX">$9.45 \times$</tex-math></inline-formula> compression at the cost of 0.8% drops in accuracy and <inline-formula><tex-math notation="LaTeX">$1.87 \times$</tex-math></inline-formula> speedup compared with the implementation on CUDA cores. Our TTN algorithm achieves up to <inline-formula><tex-math notation="LaTeX">$11.17\times$</tex-math></inline-formula> speedup compared with TensorNetwork, indicating the potential of optimized tensor learning primitives for the classical simulation of quantum machine learning algorithms.

Tensor Singular Spectrum Decomposition: Multisensor Denoising Algorithm and Application

Realising multi-sensor signal fusion and weak feature adaptive extraction is a challenging task. Therefore, a new algorithm called tensor singular spectrum decomposition is proposed in this study for the adaptive decomposition of multi-sensor time series. Traditional tensor decomposition algorithms, such as CP, HOSVD and Tucker decomposition, are derived from <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</i> -mode product. The <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</i> -mode product essentially uses the idea of matrices to deal with tensors given that it defines the multiplication between matrix and higher-order tensor, thereby creating problems of non-pseudodiagonal core tensor and nonunique decomposition results in traditional tensor decomposition algorithms. To this end, decomposition of the original tensor signal and reconstruction of multi-sensor component signals are realised in this study by combining the trajectory tensor construction, superposition of the Gaussian function spectral model, adaptive iterative optimisation of embedding dimension and diagonal average method on the basis of the principle of tensor–tensor order-preserving multiplication. The proposed algorithm inherits the perfect mathematical theory and excellent properties of matrix singular value decomposition in processing single-sensor signals whilst retaining the inherent structure and coupling relationship between multi-sensor data and realising the organic fusion and adaptive decomposition of multi-sensor signals. The analysis results of simulation, experimental and engineering signals showed that the proposed method can effectively extract weak fault quantification features hidden in original multi-sensor signals compared with existing methods.

Adaptive granularity in tensors: A quest for interpretable structure.

Data collected at very frequent intervals is usually extremely sparse and has no structure that is exploitable by modern tensor decomposition algorithms. Thus, the utility of such tensors is low, in terms of the amount of interpretable and exploitable structure that one can extract from them. In this paper, we introduce the problem of finding a tensor of adaptive aggregated granularity that can be decomposed to reveal meaningful latent concepts (structures) from datasets that, in their original form, are not amenable to tensor analysis. Such datasets fall under the broad category of sparse point processes that evolve over space and/or time. To the best of our knowledge, this is the first work that explores adaptive granularity aggregation in tensors. Furthermore, we formally define the problem and discuss different definitions of "good structure" that are in practice and show that the optimal solution is of prohibitive combinatorial complexity. Subsequently, we propose an efficient and effective greedy algorithm called ICEBREAKER, which follows a number of intuitive decision criteria that locally maximize the "goodness of structure," resulting in high-quality tensors. We evaluate our method on synthetic, semi-synthetic, and real datasets. In all the cases, our proposed method constructs tensors that have a very high structure quality.

Randomized algorithms for tensor response regression

AbstractIn this paper, we consider the estimation algorithm of tensor response on vector covariate regression model. Based on projection theory of tensor and the idea of randomized algorithm for tensor decomposition, three new algorithms named SHOLRR, RHOLRR and RSHOLRR are proposed under the low‐rank Tucker decomposition and some theoretical analyses for two randomized algorithms are also provided. To explore the nonlinear relationship between tensor response and vector covariate, we develop the KRSHOLRR algorithm based on kernel trick and RSHOLRR algorithm. Our proposed algorithms can not only guarantee high estimation accuracy but also have the advantage of fast computing speed, especially for higher‐order tensor response. Through extensive synthesized data analyses and applications to two real datasets, we demonstrate the outperformance of our proposed algorithms over the stat‐of‐art.

Tensor Multi-Clustering Parallel Intelligent Computing Method Based on Tensor Chain Decomposition.

Adaptable methods for representing higher-order data with various features and high dimensionality have been demanded by the increasing usage of multi-sensor technologies and the emergence of large data sets. Arrays of multi-dimensional data, known as tensors, can be found in a variety of applications. Standard data that depicts things from a single point of view lacks the semantic richness, utility, and complexity of multi-dimensional data. Research into multi-clustering has taken off since traditional clustering methods are unable to handle large datasets. There are three main kinds of multi-clustering algorithms: Self-weighted Multiview Clustering (SwMC), Latent Multi-view Subspace Clustering (LMSC), and Multi-view Subspace Clustering with Intactness-Aware Similarity (MSC IAS) that are explored in this paper. To evaluate their performance, we do in-depth tests on seven real-world datasets. The three most important metrics Accuracy (ACC), normalized mutual information (NMI), and purity are grouped. Furthermore, traditional Principal Component Analysis (PCA) cannot uncover hidden components within multi-dimensional data. For this purpose, tensor decomposition algorithms have been presented that are flexible in terms of constraint selection and extract more broad latent components. In this examination, we also go through the various tensor decomposition methods, with an emphasis on the issues that classical PCA is designed to solve. Various tensor models are also tested for dimensionality reduction and supervised learning applications in the experiments presented here.

Travel Patterns Analysis Using Tensor-Based Model from Large-Scale License Plate Recognition Data

Travel patterns reflect the regularity of residents’ mobility, and it is a crucial factor to evaluate the reasonability of urban spatial structure and connectivity of road networks. Therefore, exploring travel patterns is of practical significance for urban planning, traffic management, and improvement of the operational efficiency of the transportation system. In this study, we apply the tensor model to explore travel patterns under temporal and spatial dimensions based on the license plate recognition (LPR) data collected from the Changsha city, China. As travel patterns are influenced by many variables, a method framework based on the tensor model is proposed to explore the influence of variables on travel characteristics. Firstly, we apply clustering algorithms and the principal component analysis method to extract main feature variables, which can achieve the purpose of dimensionality reduction and eliminate the complex collinearity among variables. Then, the tensor decomposition and reconstruction algorithms are performed based on extracted feature variables to analyze their influence on travel patterns. The experiments demonstrate the advantages of the proposed method framework.

Two-dimensional DOA estimation of coherent signals for MIMO Radar Based on tensor decomposition

This paper studies the two-dimensional DOA estimation of coherent targets. Subspace algorithm is usually used to solve this problem. Compared with subspace algorithm, tensor decomposition has significant advantages in estimation accuracy and has been widely used in the field of signal processing. However, the research on tensor decomposition algorithm mainly focuses on incoherent targets. Therefore, we study how to use the advantages of tensor decomposition to improve the estimation accuracy of coherent targets. Firstly, we construct the echo signal model corresponding to the “Sum array” of MIMO radar, and then use the spatial smoothing to process the data to solve the problem of covariance rank deficiency. Then a third-order tensor is constructed according to the position relationship of the data. Finally, the DOA can be estimated from the decomposed tensor according to the position relationship of the data. Simulation results show that proposed algorithm has significant advantages in estimation accuracy.

Fast CP‐compression layer: Tensor CP‐decomposition to compress layers in deep learning

Deep neural network (DNN) shows its powerful performance in terms of image classification and many other applications. However, as the number of network layers increases, it brings huge pressure on devices with limited resources. In this article, a novel network compression algorithm is proposed that compresses the original network by up to about 60 times. In particular, a tensor Canonical Polyadic(CP) decomposition based algorithm is proposed to compress the weight matrix in the fully connected(FC) layer and the convolution kernel in the convolution layer. Traditional tensor decomposition algorithms are usually to first pre-train the weights, and decompose the weights, finally perform fine-tuning on the factors in the second training phase. Instead, the decomposed factors are directly updated by performing tensor CP decomposition on weight without fine-tuning. The proposed algorithm is called Fast CP-Compression Layer method in this paper. Experiments show that the proposed algorithm cannot only reduce computing time and improve compression factor but also improve accuracy on some datasets.

Online Nonnegative and Sparse Canonical Polyadic Decomposition of Fluorescence Tensors

The NonNegative Canonical Polyadic Decomposition (NN-CPD) is used in many fields such as in chemistry, biology and medicine. The data coming from these fields can be dynamic which lead to use real-time or “online” decomposition. Even though there are a variety of online tensor decomposition algorithms, the main assumption of all these algorithms is that the rank of the decomposition is known and/or does not vary over time. However this should not be the case in experimental conditions. In this work, we propose three algorithms to compute the online NN-CPD based on sparse dictionary learning for tracking chemical components in water by using a set of Emission and Excitation Matrices (EEMs) of fluorescence. The methods developed in this work is not limited to this application field and it addresses the major challenges posed by the variation of the CPD rank in real-time. First, the algorithms take into account the unknown factors and the variation of tensor rank. Second, previous extracted information are used to decompose upcoming new tensors. In addition to the development of these algorithms, one of the contributions of this paper is the real-time acquisition of fluorescence data in a semi-controlled environment. These algorithms were applied on these real datasets and compared to state of the art algorithms.

GTT: Leveraging data characteristics for guiding the tensor train decomposition

The demand for searching, querying multimedia data such as image, video and audio is omnipresent, how to effectively access data for various applications is a critical task. Nevertheless, these data usually are encoded as multi-dimensional arrays, or tensor, and traditional data mining techniques might be limited due to the curse of dimensionality. Tensor decomposition is proposed to alleviate this issue. Commonly used tensor decomposition algorithms include CP-decomposition (which seeks a diagonal core) and Tucker-decomposition (which seeks a dense core). Naturally, Tucker maintains more information, but due to the denseness of the core, it also is subject to exponential memory growth with the number of tensor modes. Tensor train (TT) decomposition addresses this problem by seeking a sequence of three-mode cores: but unfortunately, currently, there are no guidelines to select the decomposition sequence. In this paper, we propose a GTT method for guiding the tensor train in selecting the decomposition sequence. GTT leverages the data characteristics (including number of modes, length of the individual modes, density, distribution of mutual information, and distribution of entropy) as well as the target decomposition rank to pick a decomposition order that will preserve information. Experiments with various data sets demonstrate that GTT effectively guides the TT-decomposition process towards decomposition sequences that better preserve accuracy.

Context-aware road travel time estimation by coupled tensor decomposition based on trajectory data

Urban road travel time estimation and prediction on a citywide scale is a necessary and important task for recommending optimal travel paths. However, this problem has not yet been well addressed: most existing approaches face serious data sparsity issues, e.g., lack of sensor data on several road segments; and it is a difficult task to capture context patterns around the road and incorporate context-aware information into travel time estimation models. Because of this, we propose to utilize trajectory data to model road travel times as this type of data covers more urban road segments than data from traditional traffic monitoring systems. Moreover, the trajectory itself has involved both travel times and the context of road congestion. A general framework for context-aware road travel time estimation (CARTE) is then put forward. Specifically, we adopt a third-order tensor to model spatiotemporal road travel times by setting the congestion level as a third dimension. By incorporating another context-aware information, namely points of interest (POI), a coupled tensor decomposition algorithm is proposed to fill in missing data. Eventually, we propose an algorithm to calculate an ultimate two-dimensional (spatial and temporal) travel time matrix by weighting the congestion probabilities of each congestion level. The effectiveness of the CARTE was validated on two real-world datasets and it was compared to the state-of-the-art methods. The experimental results demonstrate that the proposed travel time prediction approach always achieves the best performance in terms of accuracy with different data sparsity and prediction horizons.

Block Row Kronecker-Structured Linear Systems With a Low-Rank Tensor Solution

Several problems in compressed sensing and randomized tensor decomposition can be formulated as a structured linear system with a constrained tensor as the solution. In particular, we consider block row Kronecker-structured linear systems with a low multilinear rank multilinear singular value decomposition, a low-rank canonical polyadic decomposition or a low tensor train rank tensor train constrained solution. In this paper, we provide algorithms that serve as tools for finding such solutions for a large, higher-order data tensor, given Kronecker-structured linear combinations of its entries. Consistent with the literature on compressed sensing, the number of linear combinations of entries needed to find a constrained solution is far smaller than the corresponding total number of entries in the original tensor. We derive conditions under which a multilinear singular value decomposition, canonical polyadic decomposition or tensor train solution can be retrieved from this type of structured linear systems and also derive the corresponding generic conditions. Finally, we validate our algorithms by comparing them to related randomized tensor decomposition algorithms and by reconstructing a hyperspectral image from compressed measurements.

Compression of Deep Neural Networks based on quantized tensor decomposition to implement on reconfigurable hardware platforms

Deep Neural Networks (DNNs) have been vastly and successfully employed in various artificial intelligence and machine learning applications (e.g., image processing and natural language processing). As DNNs become deeper and enclose more filters per layer, they incur high computational costs and large memory consumption to preserve their large number of parameters. Moreover, present processing platforms (e.g., CPU, GPU, and FPGA) have not enough internal memory, and hence external memory storage is needed. Hence deploying DNNs on mobile applications is difficult, considering the limited storage space, computation power, energy supply, and real-time processing requirements. In this work, using a method based on tensor decomposition, network parameters were compressed, thereby reducing access to external memory. This compression method decomposes the network layers’ weight tensor into a limited number of principal vectors such that (i) almost all the initial parameters can be retrieved, (ii) the network structure did not change, and (iii) the network quality after reproducing the parameters was almost similar to the original network in terms of detection accuracy. To optimize the realization of this method on FPGA, the tensor decomposition algorithm was modified while its convergence was not affected, and the reproduction of network parameters on FPGA was straightforward. The proposed algorithm reduced the parameters of ResNet50, VGG16, and VGG19 networks trained with Cifar10 and Cifar100 by almost 10 times.